Television succeeds in the entertainment industry because it changes every day. Viewers demand something new and the cable and TV industry continuously delivers novelty using production oriented tools and methodologies.
Interactive computer programs such as video games, entertain for much the same reason. A user makes selections based on the information content of the display and provides feedback to the computer game program which feedback alters the course of the program.
Multimedia presentations stimulate viewers' senses utilizing a number of reproduction media such as large screen video projection systems, realistic audio reproduction systems producing three dimensional wrap around sound, light shows, smoke generators, temperature controls, motion and acceleration simulators, smell generators and the like. A multiplicity of these media can be combined to give very realistic presentations. Some presentations of this nature are found in amusements park simulations which give riders a sense of actually participating in the events being reproduced in multimedia.
With the development of video on demand system such as disclosed in the aforesaid pending Applications, a need has arisen to provide menus to assist a user, inter alia, in the selection of program materials available. In a similar application, one may wish to produce a variety of electronic catalogs from which a user can select items to purchase.
In the area of education, it is often desirable to not only present information to a student in multimedia format, by which learning is enhanced, but also to obtain feedback from the student by which the adequacy of a student's learning may be assessed. The content of the material presented to the student may be altered, based on the assessment to assist the student to overcome any deficiency in the state of his knowledge.
As another example, during delivery of a Presidential speech, one may desire to obtain immediate feedback of audience reaction to various portions of the speech.
Thus, the ability to integrate interactive functionality with other media to create an integrated interactive multimedia presentations or applications is highly desirable.
U.S. Pat. No. 5,307,456 to MacKay issued Apr. 26, 1994, discloses a multimedia production and authoring system in which the system assets and resources are interconnected by a real time local area network. The system utilizes a graphical user interface to create, define, edit and order elements for use in a multimedia production. However, typical of devices of this type, no provision is made for the integration of interactive resources into the presentation. Any interactivity would need to be added after completion of the final compositing, that is after all the objects which will constitute the multimedia application are assembled and integrated into a unitary application.
U.S. Pat. No. 5,317,732 to Gerloc et al. issued May 31, 1994, and entitled System For Relocating A Multimedia Presentation On A Different Platform By Extracting A Resource Map In Order To Re-map And Relocate Resources discloses a process for relocating a plurality of multimedia presentations from one memory to another memory. The patent apparently permits elementary computer programming to occur in the form of simple math expressions which can be integrated into course work. The patent addresses "interactive capabilities" but these appear to refer to the interactions between the program developer and the source materials which are inherent in any authoring tool rather than the kind of interaction which occurs between the end user and the developed multimedia application. The patent permits certain blocks of multimedia information which may repeat upon playback to be assembled in reusable subroutines.
Production of an interactive multimedia application (such as a movie, courseware, video games, menus and the like) is normally divided into three phases: pre-production, lasting typically three to four weeks; production, typically lasting four to seven weeks and post-production, typically lasting four to five weeks. If interactivity is needed between an end user and the multimedia presentation, an additional three to four weeks for product development and another four to six weeks for systems integration and testing can be expected. The total process, therefore, can be considered to routinely consume eighteen to twenty-six weeks.
The pre-production phase typically involves signing the agreement with the customer, logic design, schedule development and rapid prototyping. Rapid prototyping typically involves creation/acquisition of placeholder assets to represent assets to be produced, an interactive story board presentation to permit viewing the application at a high level and to permit a usability test, and casting and script writing. Rapid prototyping is a development methodology which permits a customer to see the overall functionality at a high level prior to commitment to a production budget.
The production phase consists of the acquisition or creation of graphical elements, video elements, audio elements, still images, filming and the like. During this time, the actual assets to be integrated into the final product are created.
In post-production, the raw assets are edited and portions of them combined. Audio mixing, digital editing, the application of text overlays and graphical elements are all combined or composited into an integrated multimedia presentation. Typically, after compositing, if interactivity is needed, the interactive programs are produced and integrated.
Such an 18-26 week development cycle is unsuited for development of multimedia menus for video on demand service because program offerings change rapidly. The development process is also unsuited for many of the other interactive applications discussed above because the development cycle simply takes too long. Similarly, the process of refreshing an existing multimedia application such as a CD ROM catalog is slow and cumbersome. Prior art file structures for multimedia applications tend to be unsuited for integration across the different systems available from different manufacturers of computers and video equipment.
The process of adding interactivity after compositing adds substantially to the development time. Further, the integration of computer programs providing interactive functionality requires a high level of expertise typical of that required of software engineers, making the process beyond the ability of most post-production personnel. Once the interactive functionality is designed, the post-production user has no control over the content or changes to the logic that the post-production person might desire to make in response to changes in program content. Further, the post-production user has limited control over the timing and delivery of a program previously developed. Once an interactive multimedia product has been released for distribution, there is limited or no ability to reuse assets.
The inability of a post production user to control the contents of programming means that home users don't have the ability to eliminate undesirable language in programming.
Objects and Disclosure of the Invention
Accordingly, one object of the invention is to provide methods and apparatus by which a home user of video on demand or multimedia programming has the ability to eliminate undesirable language from such programming.
Another advantage of the invention is the elimination of undesirable language from an audio signal channel utilizing speech recognition.
Another advantage of the invention is the elimination of undesirable language from an audio or text source by detecting words specified by a user to be detected in the text stream and by eliminating particular words from the text stream by preventing an audio or test source from outputting information during the time when undesirable language is present.
Another advantage of the invention is the ability to selectively customize an interactive module to edit programs selected for viewing in the home to eliminate undesirable language.
These and other objects, benefits and advantages are achieved in accordance with the invention by providing apparatus for authoring multimedia assets into a final interactive multimedia application using a computer having a display and a graphical user interface. Storage, associated with the computer, is provided to store multimedia assets and program objects for performing interactive functions. One or more timelines permit a user to place icons representing said multimedia objects and program objects on a timeline using the graphical user interface at a time corresponding to when the object should be viewed during playback, thus integrating multimedia objects together with one or more of said program objects to producing an interactive multimedia application. Playback or execution of the objects represented on the timeline occurs in a time sequence indicated by their position on the timeline.
Preferably, at least one timeline is dedicated to interactive objects. Control of playback or execution of the objects is achieved using edit decision lists (EDL) and interactive decision lists (IDL) which capture the editing decisions made by a user of the tool. The interactive decision list is used to activate retrieval of objects from assets stored, to initiate playback of the objects retrieved and to initiate loading and execution of program objects all in a sequence corresponding to that represented on the timelines.
Objects can be retrieved from storage over a network. Playback or running of the interactive multimedia application can be controlled by executing an IDL at either an end user's location or at a server's location. Since the IDL is a text file, typically represented in ASCII, an interactive multimedia application can be edited using a standard text editor and items can be added, deleted or changed.
A network arrangement can be utilized for the delivery and presentation of multimedia applications represented in an edit decision list or interactive decision list. One or more file servers are connected to the network, which server contains multimedia assets. At the end user location, a set top box, such as a digital entertainment terminal (DET) is connected to the network which uses the edit decision list or interactive decision list to activate retrieval of objects stored on the one or more file servers. Playback of the objects retrieved occurs in a sequence corresponding to that represented on the edit decision list or interactive decision list. Software known as an interpreter may be used to interpret the IDL or EDL one statement at a time to generate the necessary commands to carry out the edit decision reflected in statements on the list. Playback can be controlled from the server location or from the digital entertainment terminal location by retrieving and playing of program objects or by initiating loading and execution of interactive program objects retrieved, all in a sequence corresponding to that represented on the interactive decision list.
In a network arrangement a set top box such as a digital entertainment terminal can be augmented with a personal computer (PC) linked over a data port such as a PCMCIA port of the set top box. Such a PC could run the authoring tool of the invention and utilize the multimedia assets available at one or more file servers on the network to generate multimedia applications in the PC by creating IDLs just as is done in a post production suite.
By adding text editing capability in the set top box, such as by retrieving and executing a program object, IDLs can be created and edited by the end user directly. Text editing, of course, can be performed in an intelligent terminal or DET.
Other types of program objects may be added. For example, where the interactive decision list describes a text source of information which parallels the audio associated with a multimedia application, one program object reads the text source and prevents the audio from being played back when words specified by the user in a stop list appear. In another version, speech recognition is used in lieu of a parallel text channel.
Still other objects and advantages of the present invention will become readily apparent to those skilled in this art from the following detailed description, wherein only the preferred embodiment of the invention is shown and described, simply by way of illustration of the best mode contemplated of carrying out the invention. As will be realized, the invention is capable of other and different embodiments, and its several details are capable for modifications in various obvious respects, all without departing from the invention. Accordingly, the drawing and description are to be regarded as illustrative in nature and not as restrictive.